A Comparison between Squared Error and Relative Entropy Metrics Using Several Optimization Algorithms

Convergence rat es and generalization performance are compared for the squared error metric and a relat ive entropy metric on a contiguity problem using several optimization algorithms. The relat ive entropy measure converged to a good solution slight ly more often than the squared error metric given the same distribution of init ial weights . However, where the results differed, the squared error metric converged on average more rapidly to solutions th at generalized better to the test data. These results are not in complete agreement with some results previously published.